Geometric Complexity - 1 Geometric Complexity and Minimum Description Length Principle
نویسندگان
چکیده
The question of how one should decide among competing explanations of data is at the heart of the scientific enterprise. Quantitative methods of selecting among models have been advanced over the years without an underlying theoretical framework to guide the enterprise and evaluate new developments. In this paper, we show that differential geometry provides a unified understanding of the model selection problem. Foremost among its contributions is a reconceptualization of the problem as one of counting probability distributions. This reconceptualization naturally leads to development of a "geometric” complexity measure, which turns out to be equal to the Minimum Description Length (MDL) complexity measure Rissanen (1996) recently proposed. We demonstrate an application of the geometric complexity measure to model selection in cognitive psychology, with models of cognitive modeling in three different areas (psychophysics, information integration, categorization).
منابع مشابه
Complexity Approximation Principle
We propose a new inductive principle, which we call the complexity approximation principle (CAP). This principle is a natural generalization of Rissanen’s minimum description length (MDL) principle and Wallace’s minimum message length (MML) principle and is based on the notion of predictive complexity, a recent generalization of Kolmogorov complexity. Like the MDL principle, CAP can be regarded...
متن کاملInformation Geometry and Minimum Description Length Networks
We study parametric unsupervised mixture learning. We measure the loss of intrinsic information from the observations to complex mixture models, and then to simple mixture models. We present a geometric picture, where all these representations are regarded as free points in the space of probability distributions. Based on minimum description length, we derive a simple geometric principle to lea...
متن کاملMinimum Description Length and Psychological Clustering Models
Clustering is one of the most basic and useful methods of data analysis. This chapter describes a number of powerful clustering models, developed in psychology, for representing objects using data that measure the similarities between pairs of objects. These models place few restrictions on how objects are assigned to clusters, and allow for very general measures of the similarities between obj...
متن کاملA Note on the Applied Use of MDL Approximations
An applied problem is discussed in which two nested psychological mod-els of retention are compared using minimum description length (MDL).The standard Fisher information approximation to the normalized maximum likelihood is calculated for these two models, with the result that the full model is assigned a smaller complexity, even for moderately large samples. A geometric interpretation for thi...
متن کاملThe Minimum Description Length Principle in Coding and Modeling
We review the principles of Minimum Description Length and Stochastic Complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon’s basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to ...
متن کامل